TITLE OF THE INVENTION 
INFORMATION PROCESSING APPARATUS AND METHOD 



FIELD OF THE INVENTION 
5 The present invention relates to an information 

processing apparatus and method for demultiplexing and 
decoding a bitstream, which contains one or a plurality 
of encoded object data and management information for 
managing them, in units of object data, and 
10 multiplexing and outputting one or a plurality of 
decoded object data. 

BACKGROUND OF THE INVENTION 

In recent years, as a scheme for combining 

15 multimedia data containing a plurality of object data 
such as moving picture objects, audio objects, and the 
like, and transmitting them as a single bitstream, 
so-called MPEG-4 is standardized. The receiving side 
(reproduction side) of MPEG-4 reproduces, e.g., audio 

20 and video picture scenes in association with each other 
In such MPEG-4 player, various use limitations must be 
imposed on all or some data to protect intellectual 
properties (including copyrights) . 

A bitstream specified by MPEG-4 mentioned above 

25 has a function of independently transmitting/receiving 
many video scenes and video objects on a single 
bitstream unlike a conventional multimedia stream. 



Also, as for audio data, many object data can be 
decoded from a single bitstream. That is, a bitstream 
that contains a plurality of object data like an MPEG-4 
stream contains BIFS (Binary Format for Scenes) which 
5 is specified by expanding VRML (Virtual Reality 

Modeling Language) to cover natural moving picture and 
audio data. Note that BIFS is information that 
describes MPEG-4 scenes as binary data. 

Since objects required for scene composition are " 

10 individually encoded and sent, they are also 

individually decoded on the decoding side, and a scene 
is composited and reproduced by synchronizing the time 
axes of individual data with the time axis of a player 
in accordance with the BIFS description. 

15 In this manner, since an MPEG-4 player composites 

a plurality of objects, use limitations of intellectual 
properties must be imposed on individual objects. For 
this purpose, a system for obtaining authentication 
information that pertains to intellectual property use 

20 in units of objects is proposed. 

In general, when, for example, a time limit is 
set as an example of use limitations on multimedia data 
which are distributed via a network or storage media 
such as CD-ROMs or the like, a process for appending 

25 information that pertains to the time limit (to be 
referred to as time limit information) as a file or 
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modifying data by embedding such information as a 
digital watermark is required. 

The receiving side, i.e., the player side, 
extracts the time limit information from the appended 
5 file or modified data, and limits reproduction or 

execution on the basis of the relationship between the 
extracted information and time information read from an 
internal timepiece unit of a computer or player. 

However, it is possible to set the time limit for" 
10 whole conventional multimedia data so as to prevent 

excess use, but no technique for setting a time limit 
and preventing excess use is not available for a 
specific object of multimedia data made up of a 
plurality of objects like an MPEG-4 bitstream. 

15 

SUMMARY OF THE INVENTION 
The present invention has been made to solve the 
aforementioned problems, and has as its object to 
prevent excess use of a predetermined object set with a 
20 time limit upon decoding information from a bitstream 
containing a plurality of object data. 

In order to achieve the above object, according 
to one aspect of the present invention, there is 
provided an information processing apparatus for 
25 demultiplexing and decoding a bitstream which contains 
one or a plurality of encoded object data, and 
management information for managing the encoded object 



data, and reproducing one or a plurality of decoded 
object data, comprising: 

extraction means for extracting, from the 
management information, time limit information which 
5 pertains to a time limit set for the one or plurality 
of object data; and 

control means for controlling a reproduction 
process of the one or plurality of object data on the 
basis of the time limit information. 
10 Other features and advantages of the present 

invention will be apparent from the following 
description taken in conjunction with the accompanying 
drawings, in which like reference characters designate 
the same or similar parts throughout the figures thereof 

15 

BRIEF DESCRIPTION OF THE DRAWINGS 
The accompanying drawings, which are incorporated 
in and constitute a part of the specification, 
illustrate embodiments of the invention, and together 
20 with the description, serve to explain the principles 
of the invention. 

Fig. 1 is a schematic block diagram showing the 
arrangement of an MPEG-4 player according to the first 
embodiment of the present invention; 
25 Fig. 2 is a block diagram showing an example of 

the arrangement of an IPMP controller 20 shown in 
Fig. 1; 



Fig. 3 is a flow chart showing the operation of 
the MPEG-4 player according to the first embodiment of 
the present invention; 

Fig. 4 is a flow chart showing the operation of 
5 an MPEG-4 player according to the second embodiment of 
the present invention; 

Fig. 5 is a block diagram showing the arrangement 
of a computer according to the third embodiment of the 
present invention; 
10 Fig. 6 shows an example of the data structure 

when IPMP information is appended to a plurality of 
objects in the first embodiment of the present 
invention; and 

Fig. 7 shows an example of the IPMP data 
15 structure in the first embodiment of the present 
invention. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Preferred embodiments of the present invention 
20 will be described hereinafter with reference to the 
accompanying drawings. 
[First Embodiment] 

As the first embodiment, the arrangement and 
operation of an information processing apparatus that 
25 limits reproduction of individual object data on the 

basis of information which pertains to a time limit (to 
be referred to as "time limit information" hereinafter) 
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contained in management information when a bitstream 
which contains a plurality of object data and 
management information specified by MPEG-4 is input and 
is decomposed and decoded in units of object data will 
5 be explained below. 

Fig. 1 is a schematic block diagram showing the 
arrangement of an MPEG-4 player as the information 
processing apparatus of this embodiment. The 
arrangement of the MPEG-4 player will be explained 

10 using Fig. 1, and a method of preventing excess use of 
a predetermined object set with a time limit by the 
MPEG-4 player will be explained. 

Referring to Fig. 1, reference numeral 1 denotes 
a transmission path of various networks, via which an 

15 MPEG-4 bitstream is delivered. Note that the 

transmission path 1 of this embodiment indicates not 
only communication paths such as a broadcast network, 
communication network and the like, but also storage 
media themselves such as a DVD-RAM and the like. 

20 Upon receiving an MPEG-4 stream delivered from a 

network via the transmission path 1 or an MPEG-4 stream 
read from the storage medium via the transmission path 
1, the MPEG-4 player inputs that stream to a 
demultiplexer 2. The demultiplexer 2 demultiplexes the 

25 received MPEG-4 bitstream into scene description data 
containing graphic data, audio object data (audio 
bitstream) , moving picture object data (moving picture 



bitstream) , object description data, and the like, and 
inputs them to processing circuits 3 to 6 of 
corresponding processing layers. 

Note that audio object data has been encoded 
5 (compressed) by e.g., known CELP (Code Excited Linear 
Prediction) coding, transform-domain weighted 
interleave vector quantization (TWINVQ) coding, or the 
like. Also, the moving picture object data has been 
encoded by, e.g., known MPEG-2, H-263, or the like. 

10 The object description data includes, e.g., animation 
or the like and is similarly encoded in a format 
suitable for each description data. The object 
description data is required for nodes that refer to 
audio and moving picture streams in the scene 

15 description data, and has control information (encoding 
method, correspondence with scene description, packet 
format, intellectual property management information, 
and the like) of designated streams. 

In the sync layer processing circuits 3 to 6, 

20 since the scene description data, audio objects, moving 
picture objects, object description data, and the like 
have been encoded (compressed) or the like, they are 
decoded by decoding circuits 15 to 18 in units of 
objects . 

25 Note that this embodiment assumes an apparatus 

which can decode even when the MPEG-4 bitstream 
contains a plurality of different types of objects in 



each of audio objects, moving picture objects, and 
object description data. For this reason, a plurality 
of sets of sync layer processing circuits and decoding 
circuits are prepared in correspondence with audio 
5 objects, moving picture objects, and object description 
data . 

The audio objects, moving picture objects, and 
object description data respectively decoded by the 
decoding circuits 16, 17, and 18 are input to a scene 

10 composition (device) circuit 30 to reproduce scenes, on 
the basis of scene description information decoded by 
the scene description/graphic decoding circuit 15. A 
finally obtained multimedia data sequence is supplied 
to an output device 31 represented by a display, 

15 loudspeaker, printer, or the like, and is reproduced. 

In the received data stream (MPEG-4 bitstream) in 
this embodiment, since reproduction control for 
protection of intellectual properties and the like of 
individual audio and moving picture objects that form a 

20 scene is required, intellectual property management 
information (to be referred to as IPMP (Intellectual 
Property Management and Protection) information 
hereinafter) described in, e.g., Japanese Patent 
Application No. 10-295937 is appended to the bitstream. 

25 The IPMP information is contained in the aforementioned 
object description data. Fig. 6 shows an example of 
the data structure of the overall MPEG-4 bitstream. 



Reference numeral 601 denotes BIFS mentioned above, 
which stores scene description data. Reference 
numerals 602 and 603 denote object description data, 
which describe control information for subsequent 
5 object data. In the example shown in Fig. 6, two 

object description data 602 (ODl) and 603 (0D2) are 
contained, and respectively contain information for 
controlling moving picture object data (604, 606), and 
audio object data (605, 607). Note that ODl and 0D2 
10 respectively contain IPMP information (IPMP_D1) that 
pertains to moving picture object data, and IPMP 
information (IPMP_D2) that pertains to audio object 
data . 

In the player of this embodiment, the 
15 demultiplexer 2 extracts such IPMP information and 
supplies the extracted information to an IPMP 
controller 20. The IPMP controller 20 makes 
reproduction control of individual objects by 
intercepting objects at access points 10, 11, and 12 or 
20 25, 26, and 27, superposing a disturbance signal on 
objects, decoding encrypted objects, or stopping 
reproduction of individual objects by accessing the 
decoding circuits 16, 17, and 18. In this manner, even 
when a bitstream containing copyrighted contents is 
25 input, its reproduction can be blocked. 



Fig. 2 is a block diagram showing an example of 
the arrangement of the IPMP controller 20 shown in 
Fig. 1. 

Referring to Fig. 2, reference numeral 40 denotes 
5 a CPU for arithmetic operations and control, which 
controls the IPMP controller 20; and 50, a ROM for 
storing a control program 50a for controlling the 
overall IPMP controller 20, fixed parameters 50b, and 
the like. Reference numeral 60 denotes a RAM which is 

10 used by the CPU 40 as a temporary storage, and includes 
an information area 60a for storing IPMP information, 
and a program load area 60b that stores a program 
loaded from an external storage device 70. Reference 
numeral 70 denotes an external storage device which 

15 stores a program installed using a storage medium such 
as a floppy disk, CD-ROM, or the like, and stores the 
program to be loaded onto the program load area 60b. 

Reference numeral 8 0 denotes an input interface 
which inputs IPMP information from the demultiplexer 2, 

20 and time information from the sync layers 4 to 6 . 

Reference numeral 90 denotes an output interface for 
outputting access point control signals to access 
points 10 to 12 and 25 to 27, and decoding circuit 
control signals to the decoding circuits 16 to 18. 

25 In this embodiment, the IPMP controller 20 has 

been exemplified as an independent processor element, 
but may be implemented by controlling some or all 



components of the MPEG-4 player using the CPU 40 or by 
software, as is known to those who are skilled in the 
art. 

The operation of the MPEG-4 player according to 
5 this embodiment, especially, the operation when IPMP 
information is used, will be described in detail below 
using the flow chart in Fig. 3. 

Fig. 3 is a flow chart for explaining the 
operation of the IPMP controller 20 when the MPEG-4 
10 player receives an MPEG-4 bitstream. Assume that the 
MPEG-4 bitstream contains, for example, objects moving 
picture data and IPMP information that contains time 
limit information which pertains to the moving picture 
data. 

15 In this case, the IPMP controller 20 specifies an 

encoded moving data object area demultiplexed by the 
demultiplexer 2 and sets data for one packet of that 
object as an input stream in step S300. 

It is then checked in step S301 if the input 

20 stream has reached its terminal end. If the terminal 
end of the bitstream is detected, i.e., a moving 
picture bitstream to be controlled has reached its 
terminal end, the process ends; otherwise, the flow 
advances to step S302. 

25 In step S302, a time limit information field is 

specified and extracted from an IPMP bitstream 
demultiplexed by the demultiplexer 2. 



More specifically, the IPMP information describes 
IPMP object type designation at its beginning, which 
has the following specification in FDIS of MPEG-4 
Parti : Systems : 

class IPMP_Descriptor 0 extends Base 
Descriptor : bit (8) IPMP_DescrTag{ 

bit (8) IPMP_Descriptor ID; 

unsigned int (16) IPMPS_Type; 

i f ( I PMPS_Type==0 ) { 

bit (8) URLString [size Of Instance-3] ; 

} else { 

bit (8) IPMP data [size Of Instance-3]; 




IPMPS_Type indicates the type of IPMP controller 
20; if IPMPS_Type = 0, information for protecting and 
managing intellectual properties is present at a 
location designated by the URLString. 

Values other than IPMPS_Type = 0 are registered 
in RA (Registration Authority) , and information for 
protecting and managing intellectual properties are 
described in IPMP_data. 

The IPMP controller 20 identifies IPMPS_Type in 
the IPMP information and can determine the control 
method of an object to be protected/managed by 
referring to URLString, IPMP_data, or the like. The 
syntax in IPMP_data is not standardized yet, and 



IPMP data in this embodiment is comprised of a time 
limit information field 701, access point designation 
field 702, and other management information field 703, 
as shown in Fig. 7. Note that such time limit 
5 information, access points, and various other 

management information are determined by the copyright 
holder or contents provider. 

In this embodiment, the time limit information is 
a total of browsing, display, or reproduction time 

10 periods since the first browsing, display, or 

reproduction time of the contents of a moving picture 
bitstream. Alternatively, the time limit information 
may be a predetermined time period since the first 
browsing, display, or reproduction time of a bitstream. 

15 In this case, the time limit information contains a 
time obtained by adding the predetermined period of 
time to the first browsing, display, or reproduction 
time of a bitstream. Alternatively, the time limit 
information may be a predetermined specific time, and 

20 may contain that specific time in such case. 

In step S303, it is checked with reference to the 
time limit information in an IPMP_data area extracted 
in the previous step if the time limit of a moving 
picture bitstream corresponding to the time limit 

25 information has expired. If the time limit has not 

expired yet, the flow advances to step S304; otherwise. 
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the flow advances to step S305 to limit reproduction of 
the moving picture bitstreaiti. 

The process for checking if the time limit has 
expired is executed by computing the use time period of 
the moving picture bitstream on the basis of the time 
read from a timepiece that provides a standard time via 
a network, and comparing the total use time period {the 
use time period so far + the currently computed use 
time period) of the moving picture bitstream and the 
time limit. Alternatively, when the time limit 
information is a predetermined period of time since the 
first browsing, display, or reproduction time of the 
bitstream contents, the aforementioned process is 
executed by comparing the time read from the timepiece 
with the time limit information. On the other hand, 
when the time limit information is a predetermined 
specific time, the aforementioned process is executed 
by comparing the time read from the timepiece with the 
specific time. The time source may be an internal 
timepiece of a computer that does not permit any 
tampering, or may be an internal timepiece of the 
MPEG-4 player. 

In step S304, the contents of the time limit 
information are changed if required, and only the 
required changed contents are embedded in the time 
limit information field 701 in the IPMP bitstream; if 
such change is not required, the flow advances to step 



S306 without any process. In this embodiment, since 
the time limit information is given as a total of 
browsing, display, and reproduction time periods of the 
moving picture bitstream, the elapsed time during 
5 reproduction is counted, and the result obtained by 
subtracting that counted time period from the time 
limit information is embedded as new time limit 
information in the time limit information field 701 in 
the source IPMP bitstream, thus changing (updating) the 
10 time limit of an object (moving picture object in this 
embodiment) to be controlled every time it is 
reproduced. 

The IPMP controller 20 can determine the 
reproduction control method in step S305 in accordance 
15 with the format of the IPMP information. 

In an example of the control method, access 
points are extracted from the access point designation 
field 702 of the IPMP_data area. The access point 
designation field 702 is made up of a flag 704 which 
20 points to an input stage of the decoding circuit, a 
flag 705 which points to an output stage of the 
decoding circuit, and a flag 706 which points to the 
decoding circuit itself, and an access point to be 
controlled can be specified by a flag set with a value 
25 =1. That is, an access point specifies the control 
position of the IPMP controller 20 so as to control 
decoding and reproduction of objects transmitted onto 



the transmission path 1 and, more particularly, it 
points to the input or output stages of the decoding 
circuits 16, 17, and 18, i.e., the access points 10, 11, 
and 12, or 25, 26, and 27, or the decoding circuits- 16, 
5 17, and 18. In this embodiment, since IPMP information 
is appended to a moving picture object, the access 
point in this case is one of the input stage 11 or 
output stage 2 6 of the moving picture decoding circuit, 
and moving picture decoding circuit 17. The IPMP 

10 controller 20 intercepts the bitstreara, superposes a 

disturbance signal, decodes an encrypted bitstream, or 
stops the operations of the decoding circuits at the 
designated access point, and can determine the control 
method in such case in accordance with various kinds of 

15 management information extracted from the other 

management information field 7 03 of the IPMP_data area. 
Hence, even when a moving picture bitstream containing 
copyrighted contents is input, its reproduction can be 
blocked. 

20 In step S306, the moving picture decoding circuit 

17 is started up (or its operation is continued) to 
decode the moving picture bitstream. 

In step S307, the packet of the input stream 
acquired in step S300 is released, and the flow returns 
25 to step S300 to proceed with execution. 

Note that the IPMP information is not limited to 
information that pertains to limitations on the time 



limit, but may include a combination of various kinds 
of management information for controlling the MPEG-4 
player. For example, the IPMP information may be a 
limitation of a user of this MPEG-4 player, a 
limitation on an access device (peripheral device) to 
MPEG-4, authentication information such as a descramble 
key, or the like. 

Furthermore, in this embodiment, the MPEG-4 
bitstream is comprised of a bitstream of moving picture"^ 
data in units of packets, and that of IPMP information 
of that moving picture data. In addition to the moving 
picture data, IPMP information can be appended to 
object data that form the moving picture data, and 
audio data. Hence, an object for which the time limit 
is set and which undergoes excess use control is not 
limited to the moving picture object, and the present 
invention can be applied to object data that form the 
moving picture data, and audio data. 

Therefore, the MPEG-4 player as an information 
processing apparatus of this embodiment and its control 
method can prevent excess use of a predetermined object 
set -with the time limit upon decoding information from 
a data stream containing a plurality of object streams. 
[Second Embodiment] 

In the first embodiment shown in Fig. 3, after 
the time limit information in the IPMP information is 
extracted and analyzed, reproduction control of an 



object to be controlled is made using another 
management information (IPMPS_Type) . Alternatively, 
the object of the first embodiment can also be achieved 
by the arrangement that authenticates management 
5 information other than the time limit first, and the 

operation in this case is as shown in the flow chart of 
Fig. 4. 

In such case, before extracting time limit 
information in step S404, IPMP information is 
10 authenticated with reference to IPMPS_Type and 
URLString or IPMP_data in step S402. 

Authentication is made to confirm if the user is 
authentic, has paid a fee, or the like. For example, a 
user interactive method for making the user input a 
15 password that only an authentic user knows, or insert a 
smart card may be used. 

It is checked in step S403 if the user is 
authentic. If YES in step S403, the flow advances to 
step S404 to authenticate and control the time limit 
20 information; otherwise, the flow advances to step S407 
to control reproduction. The subsequent processes are 
the same as those in the flow chart shown in Fig. 3. 

As described above, according to the arrangement 
of this embodiment that authenticates management 
25 information other than the time limit first, excess use 
of a predetermined object set with the time limit can 



be prevented upon decoding information from a data 
stream containing a plurality of object streams. 
[Third Embodiment] 

In the first and second embodiments, a method of 
5 preventing excess use of a predetermined object set 
with the time limit using the MPEG-4 player has been 
explained. Such method can also be implemented by 
software which runs on a computer. 

In this case, the program code of the flow chart 
10 shown in Fig. 3 or 4 is read from an external storage 
device onto a RAM, and the CPU executes that program 
code to implement the method. 

Fig. 5 shows the arrangement of a computer in 
this embodiment, and the arrangement will be explained 
15 below. 

Reference numeral 501 denotes a CPU for 
controlling the entire computer by executing the method 
using the program code and data stored in a RAM 503. 

Reference numeral 502 denotes a ROM which stores 
20 a startup program of the computer, setups required 

after startup, and the like, and also character codes 
and the like. 

Reference numeral 503 denotes a RAM which stores 
the program code, data, and the like loaded from an 
25 external storage device 504, and is used as a work area 
when the CPU 501 executes a process using the program 
code and data. 
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Reference numeral 504 denotes an external storage 
device which saves a program code, data, and the like 
loaded from a storage medium such as a CD-ROM, floppy 
disk, or the like. 
5 Reference numeral 505 denotes a console which 

comprises a keyboard and pointing device such as a 
mouse or the like, and allows the user to input various 
instructions to the computer. 

Reference numeral 506 denotes a loudspeaker for 
10 outputting sound of data generated by the computer 

using a PCM sound source, FM sound source, or the like. 
Also, the loudspeaker 506 can output a decoded audio 
object as sound upon decoding the audio object by the 
computer . 

15 Reference numeral 507 denotes a display unit 

which comprises a CRT display, liquid crystal display, 
or the like, can display system messages and the like 
from the computer, and can also display graphic-related 
objects contained in an MPEG-4 stream. 

20 Reference numeral 508 denotes a printer which can 

print line image data processed by the computer onto a 
paper sheet via a printer driver installed in the 
computer . 

Reference numeral 509 denotes an interface (to be 
25 abbreviated as I/F hereinafter) which can connect other 
peripheral devices (scanner, MIDI, and the like) , and 



image and audio data can be output from this computer 
via the I/F 509. 

Reference numeral 510 denotes an NCU which can 
output the aforementioned execution result of the 
5 computer onto a network or to another computer via a 

communication line. Also, the NCU 510 can receive data 
from the network or another computer. 

Reference numeral 511 denotes a bus for 

connecting the aforementioned units. 

10 With the computer which comprises the 

aforementioned arrangement, and the processing method 

of the respective unit, the computer of this embodiment 

can execute various processes in the first or second 

embodiment . 

15 [Other Embodiments] 

Furthermore, the first to third embodiments are 

not limited to only the apparatus and method for 

implementing the aforementioned embodiments, and the 

scope of the first to third embodiments includes a case 

20 achieved by supplying a program code of software for 
implementing the above embodiments to a computer (CPU 
or MRU) in a system or apparatus, and controlling to 
operate various devices by the computer of the system 
or apparatus in accordance with the program code. 

25 In this case, the program code itself of software 

implements the functions of the above-mentioned 
embodiments, and the program code itself, and means for 



supplying the program code to the computer (e.g., a 
storage medium which stores the program code) 
constitute the present invention. 

As the storage medium for storing the program 
5 code, for example, a floppy disk, hard disk, optical 
disk, magneto-optical disk, CD-ROM, magnetic tape, 
nonvolatile memory card, ROM, and the like may be used. 

Not only when the functions of the above 
embodiment are implemented by controlling various 

10 devices by the computer in accordance with the supplied 
program alone but also when the functions of the above 
embodiment are implemented by collaboration of the 
program code and an OS (operating system) running on 
the computer, another application software, or the like, 

15 such program code is included in the scope of the first 
to third embodiments. 

Furthermore, when the supplied program code is 
stored in a memory eguipped on a function extension 
board of the computer or a function extension unit 

20 connected to the computer, a CPU or the like equipped 
on the function extension board or unit executes some 
or all of actual processes on the basis of the 
instruction of that program code, and the functions of 
the above embodiment are implemented by those processes, 

25 such case is also included in the scope of the first to 
third embodiments . 



When the first to third embodiments are applied 
to the aforementioned storage medium, that storage 
medium stores the program code corresponding to the 
flow chart mentioned above (shown in Fig. 3 or 4) . - 
5 As described above, according to the above 

embodiments, excess use of a predetermined object set 
with the time limit can be prevented upon decoding 
information from a data stream containing a plurality 
of object streams. 

10 As many apparently widely different embodiments 

of the present invention can be made without departing 
from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the 
specific embodiments thereof except as defined in the 

15 appended claims. 



- 23 - 



